Evaluation of Recommender Systems Through Simulated Users

Recommender systems have proved really useful in order to handle with the information overload on the Internet. However, it is very difficult to evaluate such a personalised systems since this involves purely subjective assessments. Actually, only very few recommender systems developed over the Internet evaluate and discuss their results scientifically. The contribution of this paper is a methodology for evaluating recommender systems: the ”profile discovering procedure”. Based on a list of item evaluations previously provided by a real user, this methodology simulates the recommendation process of a recommender system over time. Besides, an extension of this methodology has been designed in order to simulate the collaboration among users. At the end of the simulations, the desired evaluation measures (precision and recall among others) are presented. This methodology and its extensions have been successfully used in the evaluation of different parameters and algorithms of a restaurant recommender system.